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METHOD FOR PROCESSING A DIGITAL IMAGE 



FIELD OF THE INVENTION 

This mventioii relates to metiiods for processing a digital image. 

BACKGROUND OF THE INVENTION 

Global distortion, or blurring, of a picture can arise ftom various factors 
5 such as distortion due to out-of-focus optics and linear translation or shaking of the 
camera during tlie exposure time. 

Blurring of a digital image may be described by means of a convolution: 

1 0 where Bo(x) is the intensity of the pixel at the address jc=(x,y) in the distorted 

picture, x bemg a two-dimeasion.al vector^ F(x) is the intensity of the pixel x in the 
undj.storted .imag^^ and h(x) is the so-called point spread function (PSF) that 
describes the distortion, Tb.e function Bo(x) is typically obtained as the output from 
a digital camera. Tlie PSF for an image distorted by out-of-focus optics, for 

15 example, can be described in a first approximation by a fimction h that is constant 
inside a circle of radius r and h(x)=0 outside the circle. 

Rectifying a distorted image involves determining tb.e function F given the 
functions Bn and k The convolution (1) can be Fourier transformed to yield 

H^)-F{q)'k{q) (2) 
20 where represents the Fourier transform- Hence, 
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F(g) = |^ (3) 

In principle, therefore, F(x) may be obtained from the inverse Fourier transform 
of Biq)fh{qy In practice, however, this solution is characterized by a very low 
signal-to-noise ratio (SNR), due to amplification of noise at firequencies q at 
which h{q) is very small. 

SUMMARY OF THE INVENTION 

In the foUowing description and set of claims, two explicitly described, 
calculable or measurable variables are considered equivalent to each other when 
tiae two variables are substantially proportional to each other. 

Tb.e present invention provides a method for rectifying a distorted digital 
image Bo(x) to produce a rectified image F(x). In accordaace with the invention, 
noise is rem,oved from the fiiD.ction Bo(q) before applying the inverse Fourier 
transform to the right side of equation (3). A noise function N is used to evaluate 
the amount of noise for functions B(q) that deviate sUghtly from Bo(q) and a new 
fimction B(q) is selected that deviates slightly from Bo(q) and that minimizes the 
noise function N . In a preferred embodiment of the invention, the noise N in an 
image B(x) is obtained based upon a calculation involving the gradient of the 
function P obtained by inverse Fouri.er transform of B{q)lh{q). In a most 
prefenred embodiment, the noise N is calculated according to the equation: 

iV= Jvp(x)-VP'(;e)i& (4) 



where "*" indicates complex conjugate. Equation (4) may be written in the 
equivalent form 
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^= j^-?il%)f ^IM^ (4) 

where D(x) is the so-called deconvolution filter (DCF) defined as Vh(x), wherein 
h(x) is the point spread function characteristic of the distortion, and q is a 
two-dimensional vector in Fourier space. The rectified image F(x) may then, be 
5 obtained by calculating the inverse Fourier Iransform of the right side of 
equation (3) using the tiius obtained B(x). However, a pattern characteristic of the 
DCF may have been superimposed on the obtained F(x). This pattern originates 
in the DCF and is correlated with it. In the most preferred embodiment, this 
pattern is removed. 

10 The invention thus concerns a method for processing a digital xm^e Bj, 

the image Bj being a convolution of an image F and a point spread function h, 
comprising removing noise from the im^ge Bj so as to produce an image B^ of 
reduced noise, and calculating F based upon B\ 

The invention further concerns a method for processing a deconvoluted 

1.5 image B, the image B having been deconvoluted according to a deconvolution filter 
D, the method comprising reducing correlation between the image and the 
deconvolution filter. 

Yet still further the invention concerns a method for processing a digital 
image BJ, the image Bi being a convolution of an image F and a point spread 

20 fonction h comprising the steps of: 

(a) removing noise from the image Bi so as to produce an image 5' of 
reduced noise; 

(b) obtaining function P,{g) according to the algebraic expression 

25 ' (c) reducing correlation between ^and 1/^so as to product a function 
P' of reduced correlation; and 
obtaining a rectified image F by inverse Fourier transform of F\q) . 
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By yet still another aspect the invention concerns a method for obtaining a 
radius r of a point spread function h describing an out-of-focus distortion of a 
digital image B, the mediod comprismg a step of calculating a gradient at a 
plurality of pixels in the image B. 



readable by machine, tangibly embodying a pro-am of instructions executable by 
the machine to perform, method steps for processing a digital image Bu the image 
Bf being a convolution of an image F and a point spread function K comprising 
removing noise &om tlie hnage Bj so as to produce an image B' of reduced noise, 

1 0 and calculating F based upon B'> 

Further the invention concerns a program storage device readable by 
machine, tangibly embodying a program of instructions executable by the 
machine to perlfonn m.ethod st^s for processing a deconvoluted image B, the 
image B having been deconvoluted according to a dcconvolution filter the 

15 method comprising reducing correlation between the image and tlae 
deconvolution filter. 

Yet stiJl further the invention concent a program storage device readable 
by machme, tangibly embodying a program of instructions eTcecutable by the 
• machine to perfomi method steps for processing a digital image Bj, the image B) 

20 being a convolution of an image F and a point spread fiin.ction h comprising the 
steps of: 

(a) removing noise from Ihe image 5; so as to produce an image & of 
reduced noise; 

(b) obtaining function P^{q) accordii^g to the algebraic expression 



5 



By a further aspect the invention concerns a program storage device 



25 



(d) 



reducing correlation between and T//jso as to product a function 
P' of reduced correlation; and 

obtaining a rectified image F by inverse Fourier transforai of F{q) . 



I -T • -T-r 
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Further the invention conceois a program storage device readable by 
machine, £aD.gibly tobodyiiig a program of instructions executable by the machine 
to perform method steps for obtaining a radius r of a point spread function h 
describing an out-of-focns distortion of a digital image the method comprising a 
stqp of calculating a gradient at a plurality of pixels in the iiD.9ge B.. 
BRJDEF DESCRIPTION OF THE DRAWINGS 

In order to understand the invention and to see how it may be carried o"ut in 
practice, a preferred embodiment will now be described, by way of non-limiting 
example oioly, with reference to the accompanying drawings, in which: 

Fig. I shows a digital image showing distortion due to out of focus optics; 

Fig- 2 shows a histogram of heights of steps in the image of Fig. 1; and 

Fig. 3 shows &e result of rectiftong the image of Fig, 1 in accordance with 
the invention. 

DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT 

In a. preferred embodiment, prior to removing noise from tlae distorted 
image Bo(x), the fimction Bo(x) is preprocessed as follows. Firstly, if the oulput 
signal Bo(x) from the digital camera is not linear with the input intensities I(x), the 
image Bri(x) is fraosfonned to make the signal linear with the intensity. This is 
accomplished by applying to the signal Bo(x) the inverse transformation that was 
applied by the camera to the intensities in order to prodnce the signal B{x), For 
example, if the camera performs a transformation on the intensity of the form 
B{x) = A{A.''^I(x)y , where A is a scaling factor and y a fixed exponent, the signal 

Bo(x) is transformed by A{B^(x)fAy'' to obtain a new B(x). For many digital 
cam.eraSj in order to make the obtained image more pleasing to the eye. The 
fimction B(x) is then transformed so as to decay smoothly at the edges to zero m 
order to malce the image periodic at the edges. The function Bi(x) produced by 
this preprocessing is operated on in accordance with the invention as described in 
detail below. 
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In accordance with the invention, a noise function N is used to evaluate 
the aniount of noise for fTractions B{q) that deviate slightly from B\{q) and a new 
function is selected that deviates slightly ftora Bx{c[) and that minimizes the 
noise function N. In a preferred embodiment of the invention, tlie noise N in an 
image S(^)is obtained based upon a calculation, involving the gradient of the 
function P obtained by inverse Fouriei: transform of B{q)/h(q). In a most 
preferred embodiment, the noise N is calculated according to the equation: 

iV= Jv?(x)-V"(;c)<fe 



or equivalently, N=ldq- q^piq)] - (4) 
where D(x) is the deconvolution filter defined as l/h(x), where h(x) is the 

point spread function characteristic of the distortion. 

A function B(x) of essentially minimal noise that deviates only slightly from 

B/(x) may be found by evaluating the variation of N with respect to B'^(q), 

{8N/dB*iq)). For example, a sequence of functions 5/(^)may be gen.erated 

iteratively by: 

~, , dN 
Bi + 1(^) = Bi(q) ~ S-^ 



dBi*(.q) 

In the most preferred embodiment, N is given by (4) so that 

Thus in the most preferred embodiment, (N given by (4)), for finite a, i=n 
0 for large n, and e=a/n, Mq) is given by: 



M?) = si(gxi-^lp(^)l r)% 



and in tile limit. 



wvi 111 ut 1 ni»i 1 
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" (6) 



(q) is preferably multiplied by a factor so that it acquires the same uorm 



5 asBiiq) to produce a function ^((j^) and a new function Pi (9)= B(q)/h{q) is then 
obtained, 

F(V may now be obtaiB.ed by inverse Fourier transform of P\(q). 
However^ a pattern characteristic of the DCF may have been superimposed on 
P\(q) . This pattern originates in the DCF and is correlated with it. In the most 

10 preferred embodiment, this pattern is removed from the function P){q) before 
applying the inverse Fourier transfomi and returning to x-space. 

In order to remove this pattern, the correlation between the pattern and the 
DCF is decreased. An overall correlation fimction C is used to evaluate tiie 
correlation between the DCF and P{q) for functions Piq) that deviate slightly 

15 from P\{q) and a new function P{q) is selected that deviates slightly from Pi(^) 
and that minimizes the correlation function C- In a preferred embodiment of titie 
invention^ the correlation C between the DCF and a function P(?)is calculated 
according to the equation: 



The variation of C with respect to P\q\ dC I dP* is used to change P(^) in 
order to reduce C. for example, a sequence of functions Pi (<?)may be generated 
iteratiyely by: 




(7) 
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In tlie mostprefeanred embodiment, C is given, by (7) so tbat 



P'- * .(9) = - ^^^TT = - ^-c?) ■||^(^)ir = ■P<9)a -H|5(?)f ) , 

Thus, in the most preferred embodiment (C given by (7)), for finite |3, i=n 
for large n, and s = p/n, 
5 Pn*tiq) = Pmi-^D(q)fr (8) 

and in the limits 

(^) is multiplied by a factor to produce a ?(^) having the same norm as 

P\(q) . is tlien obtained by inverse Fourier transfer of P{q) . 
10 After removing the superimposed pattern correction, the fiinction F(x) may 

be converted from the linear intensity range to an intensity space suited for the 
eye. However, where the original Bo(x) was near 0, the slope of Bo(x) was very 
steep, so that when values of Bo(x) near 0 are converted for discrete 
representation, much mformation is lost. Thus, despite the fact that the resolution 
15 of the data is nominally 8 bit, with dark colors (corresponding to low values of 
5), the resolution of the data is equivalent to 3 bit. 

Because much information has been lost in the dark regions, converting 
F(x) to an intensity space suited for the eye is preferably not performed simply by 
applying the inverse of the transformation previously applied by the camera to 
20 B(x) in order to convert it to the linear range. Instead, image F(x) is preferably 
first mapped into an interval, and then the' mverse transformation is applied. 
Thus, for example, if the transformation ^(5(x)/ J.)'"' was applied to B(x) by the 
camera, F(x) is mapped linearly into the interval [0,255] and the transforraatioa 
A{ Fix) /AY is applied to the fimction F(x). For the obtained image, a histogram is 
■ 25 obtained firom which a cut-off value is obtained at which a predetermmed fiaction 
of pbcels, for example 10% of the pixels in the image have an intensity value less 
than or equal to this ctit-oJEf value. The intensities are titien linearly remapped so 
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that 255 is mapped to 255, the cut-off value is mapped to 25 (10% of 255), and 
mtensity values between 0 and the cut-off are mapped to zero. 

The PSF for an image distorted by out-of-focus optics can. be described in 
a first approximation by a fimclion h tiiat is constant inside a circle of radius r and 
5 h(x)—0 outside fee circle. The radius r may be determined from a distorted image 
B(x) by the following algorithm. The algorithm maizes use of the fact that in an 
unfocused picture, boundaries (referred to herein as ''steps") are gradual and not 
abrupt, 

A step parallel to the y-axis at x=3Co of an image is described by a 
10 Heaviside function 0(x-Xo) independent of y. The mathematical description of 
the removal from focus of ihis step is then given by: 

/(x)= px'cf//z(x-x',y~j;')©(x'"Xo) 



where the intensity / in the vicinity of tb.e step is independent of y and h(x,y) is 
15 the PSF function- Integration with respect to y yields 



The slope of tlie step at x = xo is 



20 



dx 



^-yldx^®{x^-x,} 



X==Xq 



dx 



^2 



X^^Xr, 



X ^ Xn 



dx' 



^ \ dx' ) 
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= li&<JCx 5 = — = — 

In accordance with the invention, r is found by determining die slope s(x) at each 
pixel X where a step exists in the nnage B](x)^ where s is defined as the ahsointe 
value of ttie gradient of Bj(x) divided by the height of the^step at x. The radius 
5 r(x) at X is 

Edges in the image Bj(x) are detected by any method of edge detecl^on as is 
known in the art^ for example, as disclosed in Crane, R., '^Simplified Approach to 
10 linage Processing, A: Classical andModen^i Techniques in C\ Chapt^ 3. Prentice 
Hall, 1996, Calculated radii *r(x) for pixels x at an edge in the image B](x) are 
normalized by dividing by the height of the step at the edge. The normalized radii 
ai-e arranged in a histogram. The radius of the PSF is obtained essentially equal to 
the radius of maximum frequency in the histogram. 

13 

Example 

Fig. 1 shows an image that is blurred due to out-of focus optics. This image 
was rectified in accordance with the ioyention follows. 

The radius of the PSF of the distortj.on was first found as follows- Edges in 

20 the image were detected and radius r(x) was calculated for each pixel x located at 
an edge as described above. Each calculated radius was normalized by dividing it 
by the height of the step at x. Fig. 2 shows a histogram of the normaHzed heists. 
As can. be seen in Fig. 2, the radius of maximum frequency of the histogram was 
found to be 4 pixels^ and this was taken as the radius of the PSF of flie distortion. 

25 then obtain.ed according to equation (6) using a^O^OOl, and 

then normalized so as to have the same norm as {g) , A function Pi(q) was then 
obtained by applying tb.e right side of equation (3) using fi^C?) fox Bo^q) ^Ppiq) 
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was tlien obtained according to equation (9) using p^O.Ol, and then normali;&ed 
so as to have the same norm as ^(?) * The fuDcfion F(x) was then obtained by 
inverse Fourier transform of ^ {q) , The rectified image F(x) is shown in Fig. 3, 

It will also be understood tliat the system according to the invention may 
be a suitably programmed computer. Lil<:ewise, the invention contemplates a 
computer program being readable by a computer" for executing the metb^od of the 
invention. The invention fuxlher contemplates a machine-readable memory 
tangibly embodying a program of instructions executable by the machine for 
executing the method of the invention. 



